Data Merge Process

Summary

The data merge process at AANP is critical for maintaining data integrity by identifying and merging duplicate records. This process, which involves both automated and manual reviews, ensures the accuracy and consistency of member and contact information within Salesforce. The document outlines the requirements for enhancing the data importing and de-duplication processes, with a focus on streamlining workflows, automating tasks, and providing tools for efficient data management.

Currently, there are over 6,000 records in the merge queue, with an average of 100 merges processed per month. The goal is to automate and optimize these processes, ensuring that the workload is manageable and that data remains accurate and up-to-date.

Key Components

1. Automated Matching Logic:

   • Implement a matching logic using criteria such as NPI number, NCSBN ID, email address, phone number, and mailing address, alongside names (first, middle and last).
   • Introduce a tiered matching system to prioritize exact matches on critical fields, including an percentage match indicator.
   • Ensure that new records cannot be created with the same email address unless no email is provided in the existing record.
   • Duplication warnings to the end user should follow a predefined order and provide generic warnings without specifying the type of match.

2. Manual Review Process:

   • Allow internal users to review flagged potential duplicates to verify matches.
   • Enable internal users to select the origin of specific fields that will survive for the final merged record.
   • Maintain an audit trail of all changes made during the merge process and write those changes to the surviving accountin the notes.

3. Address Validation and Normalization:

   • Ensure consistent address formatting by leveraging the existing Google Address API integration.
   • Extend the Google integration to include address normalization and capture additional metadata such as assigned AANP regional directors, state liasions, along with their county, governmental state representatives, and state senators.

4. Audit Trails and Metrics:

   • Track all changes and maintain detailed audit trails for each merge and note changes in the surviving account.
   • Monitor key metrics such as the number of merges, reasons for merges, and who completed them. Create a merge report that can be run for a speicifc date range indicating all details mentioned.

Internal User Requirements

Identification and Matching

1. 1. As an internal user, I should be able to identify potential duplicate records based on matching criteria such as NPI number, NCSBN ID, email address, phone number, mailing address, and names.
2. As an internal user, I should be able to view a list of these potential duplicates.
3. As an internal user, I should be able to easily compare multiple data points from each potential duplicate to resolve difficult matches through the UI and/or reports.
4. As an internal user, I should be able to validate data quality during the merge process to catch errors early.
5. As an salesforce administrator, I should be able to adjust matching criteria weights and logic as patterns change over time.
6. As an internal user, I should be alerted to potential duplicates with help info and reason for message.
7. As an salesforce administrator, I should be able to specify how generic duplication warnings will be communicated to end users, ensuring users understand potential duplicates without confusion.
8. As an internal user, I should be able to view individual lists of duplicate contact records at a specific date or date range using the matching criteria provided.

Review and Merge Process

1. As an internal user, I should be able to manually review flagged potential duplicates to verify matches.
2. As an internal user, I should be able to search for contact records by name, ID or email address to merge from a list or search by member ID to merge from a list (2 or 4 records).
3. As an internal user, I should be able to select a winning/surving record to default data selection on specified fields such as MemberType and MemberType ID, active membership, community subscriptions, other business rules such as Fellow status, Committee service, order history). Otherwise, desired membership and subscription rules may not be reflected properly on surviving member record.
4. As an internal user, I should be able to view/compare certain data fields for a merge transaction but am restricted from modifying selected data fields that are presented (ex: IsFellow__c, IsSPGMember__AANP, AANP General (profile) and other sub-type records that exist, etc.)
5. As an internal user, I should be able to select the field value to include in the merged record, even if that value is not from the winning/surviving record.
6. As an internal user, I should be able to submit a merge request and automatically keep the most recent info entered by member in certain fields that I specify including their AANP profile (ex: profession or salary) and not use winning record data by default.
7. As an internal user, I should be able to submit a merge request and automatically retain email addresses (web users) and merge those web user records into specified fields depending on number of contact records being merged (i.e. move non-surviving data into secondary or tertiary email fields of winning one).
8. As an internal user, I should be able to track all account changes over time for auditing purposes.
9. As an internal user, I should be able to streamline the review and completion of merges.
10. As an internal user, I should be able to reference historical snapshots of records for auditing or research purposes.
11. As an internal user, I should be able to generate tasks for IT Support if an error occurs during the merge process.
12. As an internal user, I should be able to view the manual merge activity within a member’s contact record (who merged, what changed, when did it occur) for member support troubleshooting.
13. As an internal user, I should be able to view a report of successful or failed merges for a specific date range.

Automated Merging

1. As an internal user, I should be able to run customized reports on key metrics like the number of merges, reasons for merges, completion details, and who completed them and the timeframe.
2. As an internal user, I should be able to schedule an automated merging process according to specified criteria to run at specific intervals.
3. As an internal user, I should be able to define the automated merging criteria, such as names, locations, licenses, etc., for perfect matches.
4. As an internal user, I should be able to test new matching rule scenarios before implementing live changes.
5. As an internal user, I should be able run automated merging process that chooses a winning record and will default data selection on specified fields such as MemberType and MemberType ID, active membership, community subscriptions, other business rules). Otherwise, desired membership and subscription rules may not be reflected properly on surviving member record.
6. As an internal user, I should be able to implement logic to retain email addresses (web users) and merge those web user/email address records depending on number of contact records being merged (move non-surviving data into secondary or tertiary email fields).
7. As an internal user, I should be able to view automated merge activity from a member’s contact record (what job, what records, when did it occur).

Metrics and Reporting

1. As an internal user, I should be able to run customized reports on key metrics like the number of merges, reasons for merges, completion details, and who completed them.
2. As an internal user, I should be able to easily view trend metrics on match rates and manual work over time.
3. As an internal user, I should be able to generate a weekly report for the CE Center vendor (InReach) that provides a list of merged records and all necessary data.
4. As an internal user, I should be able to schedule an automated process that delivers a weekly report to the CE Center vendor (InReach) with a list of merged records and necessary data.
5. As an internal user, I should be able to schedule an automated process that adds successfully merged records to the AANP legacy database merge queue table.
6. As an internal user, I should be able to regularly monitor and assess the performance of automated matching logic and address validation processes to identify and rectify any issues promptly.

Merge Requests and Management

1. As an internal user, I should be able to submit a form to the membership team to request a merge through Salesforce Service Cloud or Zendesk.
2. As an internal user, I should be able to manage a queue of merge requests from support staff or external user reports.
3. As an internal user, I should be able to handle merges related to membership issues and prioritize them accordingly with low, medium or high priority.
4. As an internal user, I should be able to undo a merge if an error occurs or if the merge was incorrect. There could be a time limit on this action.

External User Requirements

Address Validation

1. As an external user, I should be able to have my address validated and normalized upon update, ensuring consistency and accuracy.

Duplicate Record Prevention

1. As an external user, I should not be able to create a duplicate record based on my email address, ensuring that each email address in the system is unique.

Assumptions

• All internal users are included in the Personas definition.
• Timelines will follow as outlined in each requirement.

References

1. AANP Merge Handbook created by Chris Tracy
2. Internal Merge Training Video
3. Membership Merge Request Form
4. VIDEO - I2C Discovery Session
5. VIDEO - AANP Application Walkthrough
6. External website used to research NPs for validating duplicates:
   • NPPES NPI Registry
   • Nursys

---

Approvals / Signoffs

Role	Name	Signature	Date
Technical Lead	Jeff Sikes
Stakeholder	Liz Messner